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ABSTRACT 

1 As part of the Phase II report of the Illinois 

Occupational Curriculum Project (IOCP) , this document summarizes the 
investigative activities and resulting implications of both Phase I 
and Phase II. Based on the review of literature, conclusions and 
implications were stated which included; (1) The alternative 
strategies for curriculum development available in the literature may 
be broadly divided into Tylerian Models, Systems Models, and Product 
Development Models, (2) The state of the art in curriculum 
development presently evidences little forecasting power as a 
consequence of the absence of sound scientific of technical 
theorizing, (3) The development of any evaluation system from among 
the suggested prototypes should provide the user with flexibility and 
the opportunity to attend to alternative evaluate procedures, (4) The 
development of the IOCP Model should attempt to consider the critical 
aspects of various theories and philosophies of curriculum 
development, and (5) The IOCP model should attend to the human 
problems involved in implementation. Phase III of this project will 
be devoted to pilot testing the model- Related documents are 
available as VT 014 774 and VT 014 775 in this issue, and ED 050 270- 
(JS) 
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PREFACE 



This report focuses on one area of emphasis undertaken by the 
Illinois Occupational Curriculum Project in developing a model for 
occupational curriculum development and evaluation. It is only a part 
of the total Phase II report on the research and development project 
entitled The Illinois Occupational Curriculum Project, heretofore 
referred to as the Research and Development Project in Occupational 
Education entitled "The Development of Process Models for Decision- 
]Making in Curriculum Development and Evaluation." This project is 
currently in progress at Joliet Junior College, Joliet, Illinois, 
with present efforts directed toward the initial development of a 
systems model designed to assist administrators in decision-making 
related to the development and evaluation of occupational education 
programs. The project is funded by the State Board of Vocational 
Education and Rehabilitation, Division of Vocational and Technical 
Education, Research and Development Unit, State of Illinois. 



Purpose of the Project 

This project is based on the assumption that more systematic means 
must be developed to assist curriculum planners in the development of 
new programs and the continuous evaluation of on— going programs in 
occupational education. 

The following questions serve as the basis for the project' research 
and development activities: 
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1. Can generalizable systems models be developed to provide 
curriculum planners with a systematic decision-making procedure 
for program identification, development, implementation, and 
evaluation? 

2. Is it possible to develop guidelines for the identification and 
utilization of resources and evaluative criteria in accomplishing 
the activities specified in the systems model? 

Objectives of the Project 

The following are the overall project objectives: 

1. To develop systems models for curriculum development and 
evaluation in occupational education. 

2. To develop guidelines for the utilization and application of the 
systems models. 

3. To test the applicability and usefulness of the systems models 
in a pilot situation at selected institutions offering 
occupational programs. 

4. To develop a plan for dissemination and in-service training 
for curriculum planners in the utilization of the systems 
models . 

5. To promote research in related areas. 

Overview of the Total Project 

The project is divided into four distinct phases. These are: 

Phase I: Project Planning 

Phase II: Initial Systems Model Development and Preliminary 

Evaluation 



Phase III: 



Pilot Testing of the Model 



Phase IV: 



In-depth Evaluation of the Project and Dissemination 
of the Findings 

Phase I focused on a review of the literature, while Phase II 
involved the comparison and evaluation of systems, models, end decision- 
making and the development of a systems model for curriculum development 
and evaluation in occupational education. Phase III and Phase IV are 
proposed for further development, implementation, and evaluation of the 

model. 

Phase I: Project Planning 

Phase I was initiated March 1, 1970, with a grant of $24,550.00 
from the State Board of Vocational Education and Rehabilitation. This 
grant combined with $6,916.00 in local funds providing a total budget 
of $31,466.00 to conduct the project through June 30, 1970. 

The project planning activities centered around three major areas of 
concern identified as being particularly important to the establishment 
of a firm basis for the project: 

1. Review of the literature on models for curriculum development 
and evaluation. 

2. Review of current thinking on the effects of planned curriculum 

on soc ial and economic conditions. 

3. Study of potential consultants and resources agencies qualified 

to assist in subsequent phases of the project. 

P has e II: Initial Systems Model Development 

And Preliminary Evaluation 

Phase II was initiated July 1, 1970, with a grant of $67,178.00 
from the State Board of Vocational Education and Rehabilitation. This 
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grant combined with $16,950.00 in local funds providing a total budget 
of $84,128.00 to conduct the project through June 30, 1971. 

This phase of the project focused on research and development 
activities in four mrjor areas of concern directed toward the initial 
development and validation of a systems model for curriculum development 
and evaluation in occupational education. The following topics served 
as the focus of investigative activities for Phase II of the project: 

1. Investigation of Management Systems 

2. Investigation of Curriculum Models 

3. Identification of Decision-making Practices in Occupational 
Education 

4. Initial Model Development and Testing 

Developmental efforts were executed to coordinate the findings from 
the aforementioned areas of investigation with the objective of developing 
an initial systems model for decision-making in curriculum development 
and evaluation. 



Future Phases of the Project 

Two additional phases of this project are planned. Upon completion 
of Phase TI, Phase III is proposed for pilot testing the model. This 
pilot test will provide orientation workshops for the application and 
use of the model, field testing of the model under actual conditions, 
and implementation of the model in selected institutions. Phase IV will 
provide for an in— depth evaluation of the project and the dissemination 
of findings to other institutions for their use in developing and 
evaluating occupational curricula. 
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CHAPTER I 



INTRODUCTION 

The funding of the Illinois Occupational Curriculum Project (IOC?) 
Is based on the assumption that a more systematic means can be developed 
to assist curriculum planners in the development of new programs and 
the evaluation of on-going programs in occupational education. Saving 
based a research and development effort on this assumption made clear 
the need to complete an Investigation of curriculum development and 
evaluation models already in use or in the literature. 

This report summarizes the investigative activities and Implications 
drawn from a number of months of literature search and discussions with 
consultants concerning existing curriculum development and evaluation 
models. This investigation spanned both Phase I (Project Planning) and 
Phase II (Initial Model Development) of the IOC?. 

To guide this investigation, a series of questions were identified, 
including the following: 

1. To what extent is it feasible to combine parts (variables, 
decisions, contents, etc.) of the three general types of models 
identified in Phase I and any other that may be identified in 
the earlier parrs of Phase II? The purpose of this combination 
would be to make a more adaptable general model that would be 
more universally feasible. 

2. What are the variables considered in each model? 
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3. What are the decisions to be made? by whom? 

4. How can the model account for the institutional or administrative 
attitude? 

5. What expertise is needed to implement the model? 

6. Where does responsibility for success in curriculum development 
and evaluation lie? 

7. What is the criteria for identifying variables. 

8. What is the criteria for identifying and classifying parts of 

models? 

9. What is the extent of support the models lend each other and 
the overlap that they have? 

10. To what extent can development and evaluation models be 
combined? 

Both in Phase I and Phase II, activities related to this investigation 
were executed with a good deal of consultative assistance. The use of 
consultants proved advantageous, for reasons of expertise and time. The 
ambitious nature of the Illinois Occupational Curriculum Project 
prohibited project staff from spending long periods of time on research 
of this nature. Rather, project staff time was devoted more to the 
analysis and synthesis of data gathered and ascertaining the implications 
for the development of a systems model for occupational curriculum 
development end evaluation • 

following is a list of terms defined as used in this report: 
Curriculum - that set of educational activities under the 
control of the college . (34) 

Model - a conceptualization in the form of an equation, a 
physical device, a narrative or graphic analog representing a 
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real-life situation. (59.) 

3. Occupational Curriculum - that set of experiences and activities 
aimed at preparation for employment. (34) 

4. Program - a subpart of the curriculum; the combined set of 
activities and experiences of a particular individual in 
preparation for specific results. 

5. System - the structure or organization of an orderly whole, 
clearly showing the interrelations of the parts to each other 
and to the whole itself. (59) 



CHAPTER IT 
REVIEW OF THE LITERATURE 



The following report is an in-depth analysis of the investigative 
activities conducted concerning models of curriculum development and 
evaluation. The consideration of each development process was guided by 
the following set of questions. These questions, as presented he 
used as analytic guides in the consideration and presentation of each 

model . 

1. Who authored the model, and what has been the extent of its 

documentation? 

2. What assumptions underlie the model, and are they enumerated 
in a rationale? 

3. What are the major components and/or phases of the model? 

4. Does the model provide substantive illustrations or are they 

available elsewhere? 

5. Does the model contain components that would qualify it for 
one type of development activity rather than another? 

6. At what levels of specificity* does the model function? 

^ T-ow (A \ __ Model is basically composed of broad 

*Levels of Specificity. Low UU , ef .. nl - tlons Q f its major components 
verbal and graphic outlines an<l/or ^ ^ contains descriptions 

and phases; Middle (B) Ili a between the several continuum on 

or explanations of the relations^ps betwe^ ^se^ ^ ^ ^ ^ ^ 

p^L^t^^liSi^catious of Cashs or subsystems and 

indications of parameter locations* 
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Models for Curriculum Development 

As a result of the review of the literature, three basic approaches 
to curriculum development were identified: (1) The Objectives Approach; 

(2) The Product Development Approach; and (3) The Systems Approach. Some 
examples of these three basic types of curriculum development models are 
presented herein, outlined using the format of the above six questions. 

Objectives Model of Curriculum Development 

1. The objectives model of curriculum development is thought to 
have originated with the work of Tyler (81) . This general 
model has gained widespread acceptance. One clear delineation 
of the objectives model has been offered by Taba (77) • Taba s 
statements will be considered representative of objectives 
models. 

2. The rationale for the objectives model of curriculum development 
has been discussed in great detail by Tyler, Taba, and ochers. 
Taba (77) identified five major decisions to be made in 
curriculum development. These decisions reflect primary a ^eas 

of concern for the developer. 

a. What are the aims of the school and the objectives of 
instruction? The objectives model assumes the primacy 
of objectives in the development process. 

b. What areas or subjects are to be selected? What 
specific content is to be covered in each? 

c. What types of learning experiences are to be utilized 
in the curriculum? 

d. How is the curriculum to be evaluated? 

e. What is to be the overall pattern of the curriculum? 

O 
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The curriculum developer, then, must consider each of these 
questions as he seeks to create a curriculum. At each point, 
decisions among possible alternatives must be made. Tab- (77) 
suggests three general questions, the answers to which provide 
criteria by which the developer makes decisions. These general 
questions suggest the rationale which guides the conception of 
the objectives model of curriculum development. 

a. What are the demands, and the requirements, of the 
culture in which the curriculum will operate? 

b. What do we know about the learning process and the 
nature of the learner? 

c. What is the nature of knowledge? What are the 
characteristics and contributions of the disciplines? 

in general, the rationale for the objectives model suggests that 
curricula originate from the demands and requirements of the 
society, that the curricula ought to be firmly grounded in our 
knowledge of the learner and the learning process, and that 
the curricula ought to reflect an understanding of the nature of 
knowledge. Further, the objectives of the curriculum must be 
clearly delineated, and a means of evaluating the effectiveness 
of the curriculum in meeting the objectives must be defined as 

a part of the development process. 

3. Taba (77) suggested seven basic steps in the curriculum 

development process: 

a. Diagnosis of needs. The curriculum developer must 
assess the needs of the society. 
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b. Formulation of objectives. From the needs assessment, 
the developer formulates objectives for his curriculum. 

The objectives reflect the intent of the curriculum 

to meet identified needs. |_There is a considerable 
amount of discussion as to how objectives are to be 
stated. The primary concern is whether objectives 
must be stated in behavioral terms. For a discussion 
of various viewpoints on this issue, see Atkin (3), 

Popham, et al . (50) .J£ 

c. Selection of content. In most instances, curriculum 
developers must select representative content from a 
larger universe of possible content. The selection of 
content is closely associated with the needs and 
objectives identified previously. 

d. Organization of content- Once content is selected, 
it must be organized in some manner. Questions of 
scope, sequence, etc., must be attended to at this step. 

e. Selection of learning experiences. From the variety of 
learning experiences potentially available, the curriculum 
developer must select those that seem most appropriate 

to the objectives and the content selected for inclusion 
in the curriculum. 

f. Organization of learning experiences. As with content, 
learning experiences must be organized in some fashion. 

g. Determination of what and how to evaluate. As a part 
of the development process, the developer determines 
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what he will evaluate and how he will carry out that 
evaluation. A later section of this report deals with 
two evaluation procedures. 

4. Perhaps the most detailed illustration of the objectives model 
of curriculum development is found in Taba's .(77) work. The 
model has been used extensively. 

5. This (objectives model) is one of the most general models 
proposed. The stages in its development are generalizable to 
a wide variety of instances. 

6. The objectives model has a low level of specificity which means 
it is quite general. There are, however, many explicit 
applications and discussions of the model. 

A review of the literature reveals a vast number of sources that 
are relevant to the objectives model of curriculum development. Xn the 
literature, the Tyler entry (80), together with the Taba entry (77), 
constitute two of the most important statements about the objectives 
modol. The Douglass entry (17) is merely representative of many such 
works. 

Product Development Model of Curriculum Development 

1. The authorship of this pervasive development format is not 

attributable to a single individual or group. Its origins and 
principal proponents are in the operant psychology of B. F. 
Skinner (61), the programmed instruction movement (43), and 
the works of Tyler (82), Popham and Husek (51), Mager (41), 
and Baker (6) . 
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2. This approach has traditionally assumed that: 

a. empirically validated curriculum should be developed 
and that this process is feasible; 

b. the development program must be marked by a cyclic 
process of redefinition; ana 

c. a high degree of technical competence, facilities, and 
organization will characterize or be available to the 
development agency* 

3. The major components or phases of this model include the 

following : 

a. Formulation 

1) Description of general intents. Completion of a 
program rationale. 

2) Exploration of various sources of program goals. 
Sources includes 

a) the society and co m m un ity 

b) the institution 

c) the teacher and learner 

3) Justification of product. Search for existing 
materials and procedures that have proven effective. 

4) External review of procedures and findings (to be 
repeated throughout the development process). 

b. Specification 

1) Develop tentative, detailed specifications of project 
outcomes in terms of performance and statements of 
post— instructional behaviors for both student and 
teacher . 

O 
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2) Analysis and subdivision of more complex program 
objectives into prerequisite and component skills. 

3) Design criterion referenced items to measure 
objectives. Develop examinations containing 
measures of sub- and terminal objectives and field 
test to determine appropriate item format for target 
population. 

4) Compose tentative list of expected entering 
behaviors . 

5) Conduct a complete external review, 
c . Development 

1) Describe and produce alternative modes for presenting 
instruction. Criteria for mode selection include: 

a) replicability 

b) cost 

c) feasibility 

2) Testing of sample instructional segments. 

3) Selection of segments to be included. 

4) Statement of criteria for selection of learning 
experiences- Criteria could include: 

a) presence of practice 

b) presence of appropriate cues 

c) provision for knowledge of results 

5) Testing of longer sequences of materials on 
appropriate groups (individuals, small, large, etc.). 



O 
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d. Field Testing: Purposes 

1) To determine the appropriateness of procedures in 

real classrooms - 

2) To collect teacher observations. 

3 > To collect data on change in student behaviors or 
competencies . 

4) To/ experimentally compare alternative modes of 
presentation. 

e. Revision Cycles 

1) Organization of all sources of data: 

a) observer records 

b) user reports and preferences 

c) pupil performance 

d) results of controlled variation studies 

2) Repeat revision and field testing. Utilization of 
a cost-effectiveness criterion. 

f . Implementation 

1) Broad scale introduction to regular classroom use. 

2) Summative evaluation. 

4. Substantive illustrations of this development process are 
widely available. They represent the process in whole or part. 

i 

The citations at the end of this section present explicit 
delineations of the process or its parts- 

5. This model is most appropriate for use by a well-coordinated, 

highly trained network of product development expertise. As 
Baker (4:17-18) has suggested: ". . . the systematic development 

of curricula according to the described pattern (product 

O 
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development model) Is an exhausting and resource- draining 
enterprise. Some university-developed curricula have been 
heavily data -based* but even In eras of liberal federal funding, 
the careful management of trained development personnel has 
usually not characterized such ventures.” 

6. This model is available with a high degree of specificity - 

Sy stems Analysis and Curriculum Development 
There has been increased attention given to systems analysis for 
possible application to curriculum development . In this section, three 
systems models are presented. Each model assumes , for the most part, that 
the developmental process is linear. 

A systems approach to management cannot readily be introduced 
piece— meal into an organization. As will be shown, it would be difficult 
to use a systems approach for the development of the curricu lum while 
other aspects of college management followed conventional line and staff 
relationships. Most relevant to the practitioner in educational 
administration is simply the systems perspective. It is a way of 
thinking about management problems. 

Systems thinking will force the administrator to look at the totality 
of situations or problems, to take a long range view regarding his 
organization, to analyze consciously antecedent conditions and possible 
effects, to utilize cost— utility approaches to choice, and^to optimize 
for the total organization. The predictive power of the educational 
manager will be enhanced through a more skillful approach and an improved 
ability to deal with uncertainty. Generally, the many heuristic vehicles, 
procedures , models* and tools employed by the systems approach can 
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contribute to the facilitation of administrative practice. The approach 
must be considered ns a facilitator of the management process and not as 



a panacea* 

The systems approach can be classified as being a way of thinking 
that represents an extension of the scientific attitude and method to 
the handling of administrative problem-solving. It encourages, even 
demands, the expansion of analytical activity, and attempts to utilise 
cross-disciplinary methods. It is holistic, rather than atomistic, 
and contextual: the focus is on the total problem and all relevant parts 

as well as on the environmental context against which the problem appears. 

There are three major phases to the systems approach. These phases, 
while they appear separate in exposition, are thoroughly intertwined 



and integrated in practice. 

1. O— • ton-lysis - Systems analysis is undertaken for the 

purpose of identifying rational decisions concerning the 
design, selection, and operation of a system. The main goal 
is the identification of the one best system (and subsystems) 
and the most efficient way of operating it. Here, a clear 
distinction must be made between the process and the structur e 
of systems analysis. Process is parent to the structure. The 
analysis then sets the grand design pattern for the organization 
and in connection with the problems which will be processed. 

2. Envineering . Where a task is extensive and complex, 

there might he too many goals for a single group to manage 
properly. The task must be subdivided and assigned to several 
groups. Systems engineering divides the overall task into 
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subtasks. Assignments are then made to various groups so that 
each can operate in a well-defined sphere and where interaction 
among groups is clear-cut and minimal. A measure of the 
effectiveness of systems engineering is when the total task 
has been completed and the work of groups can be readily 
integrated into an overall working system. For example, a 
radio receiver is an operational system consisting of several 
subsystems — detector, rf, if* and af stages. Each subsystem 
has unique specifications and each must integrate with the 
other and contribute to the operation of the radio. 

3 . Sys terns Management . Frequently , management is organized along 
departmental hierarchies . Information and authorization flow 
vertically within each hierarchy • lateral flow between hierarchieo 
however, occurs only at the top. When sophisticated and complex 
activities which involve several departments of an organization 
are undertaken, the efforts of each department must be coordinated 
with the other. Management must transcend departmental 
boundaries. An important attribute of the systems approach is 
organizational control exercised by the systems manager. His 
responsibility cuts across functional and boundary lines. Here, 
authority and responsibility exist to implement the findings of 
systems analysis - 

The systems approach to management has several advantages. Xt has 
provided an avenue for functional analysis in terms of antecedent 
conditions and developmental trends- Phenomena are assessed in context, 
spatially and chronologically. Xt has provided an approach to structural 
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analysis in terms of connections and relationships. Structures are not, 
therefore, abstracted or superimposed, but are analyzed through empirical 
referents. The approach is operational. A system problem is not mechanical 
or psychological, or sociological; rather these are ways of looking at 
the problem. Problem-solving becomes a matter of looking at the system 
and the forces affecting it, and then asking and finding the answers to 
the right questions. The systems perspective is futuristic; i.e. one 
that projects developmentally long range plans. Systems thinking is a 
realistic way of manipulating variables in a complex context. End results 
are viewed in terms of relevant conditions and ultimate pay-offs. It has 
provided a unifying force for practice and inquiry and spans a number of 
disciplines. In this sense, it has resulted in a cross-disciplinary 
approach that has yielded a heuristic perspective on reality. 

Disadvantages incident to the use of the systems approach are related 
to the size of the using organization. Most administrative personnel 
have been trained in operational activities and not in the use of systems 
management . The main ingredients of the systems approach to management 
are long-term planning and research data and the technology for employing 
the ingredients. Thus, in order to introduce the systems approach, new 
personnel would have to be employed. A key person in the support staff 
is the systems analyst who would be responsible for the operation of the 

entire system and its subsystems* 

Three different systems models are presented below by source, 

documentation, assumptions, and major features. 

Systems Model Number 1 

1. Walter M. Arnold, Vocational, T^hnical. and Continuing Education 
in Pennsylvania: A System s Approach to State-Local Progra m 

0 
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Planning . Pennsylvania Department of Public Education, 1969. 

The project was undertaken as an effort to systematize state- 
wide educational planning. 

2. Several assumptions appear! 

a. There is a relationship between socio-economic planning 

and vocational education program planning. 

b. State-level planning can be integrated with that of 
local school district planning- 

c* The planning sequence is linear- 

3. Major planning steps and plan development levels are: (See Figure 1) 



Planning Steps 




pi^n Development Levels 


a . Problem Def ining 


a. 


Socio-Economic Planning 


1) objectives 


b. 


Vocational Education 


2) constraints 




Program Planning 


3) translation 


c - 


Vocational Education 


b . Problem Solving 




Resources Planning 


1) analysis 




* 


2) trade-offs 






3) synthesis 






While the model does not include 


substantive illustrations. 



ample data can be found in the literature related to Planning, 

Programming, Budgeting Systems (PPBS) - 

5. The model design appears to he an adaptation of the PPBS approach 
and modified for use in an educational system. This particular 
design, however, seems to be geared for use at a state-level 
operation. 

6. The model is of a middle level of specificity. (See Figure 1) 
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Systems Model Humber 2 



1- Stanley Young, Professor of Management, School of 

Administration, University of Massachusetts, and Cha^ 10 s 2 '* 
Summer, Columbia University, Consulting Editor to 
Foresman and Company, Management: A Systems Analysis 

Illinois: Scott— Foresman and Company, 1966. 

Search of professional literature failed to reveal ^cp* 1 ®****^^ 
2. Assumptions -which the author appears to make about t^ 
making model are: 

a. Specific recommendations for achieving total 
of a decision-making system have not yet be^ 

1) Decision-making is synonymous with proble^ ' 

2) A decision-making model is a construct wh^ c ^ 
shows hew decisions might be made. 

3) Decisions are made at each stage as a pro^^jj ^ ^ 
through a system. 

4) Decision-making can be approached through 
< ^ sc fpll nes j i.e. statistics, economics, 
sociology, psychology, etc. 

b. The proposed model is linear in the gamp sens^ f$i& c 
problem solving techniques are linear. Probl^ 
generally is sequent i al in nature — raising tha 
problem ... search for solutions ... impl^^t^' 1 ' 0 ** 

- - - etc. 

c. The proposed model is a partially closed syst^ 

d. Any decision-making model must be congruent 
management organizational philosophy. 
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3. Major steps of the model and suggested techniques appropriate 
to each step follow: 



a. Definition of Organ- 
izational objectives 


Welfare, utility, benefit, or 
value measurement theory 


b. Raising the problem 


Sampling theory and reliability 
analysis 


c. Isolating determinants 


Correlation — partial or 
multiple, regression analysis, 
factor analysis, model 
building, controlled laboratory 
experiments , historical 
analysis, personal estimation, 
logical deduction 


d. Search for solution 


Search theory, heuristics, 
information theory, 
programming — linear and non- 
linear, simulation 


e. Selection for best 
solution 


Simulation, heuristics, 
programming — dynamic, 
invention, probability theory , 
sampling theory 


f . Consensus 


Group dynamics, information 
theory 


g . Authorization 


Theory of risk 


h . implementation 


Critical path, PERT 


i. Direction 


Cybernetics, servo-theory, 
sampling theory 


j . Auditing 


Sampling theory, reliability, 
servo-theory, information 



theory 



4. The author detailed a four and a half year participation in 

the design and installation of an actual decision-nakiag system 
for a 250-bed general hospital. This was a case study uhich 
demonstrated the feasibility of planning, installing, and 
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controlling a planned decision-making system which was designed 
in terms of management problems that emerged. 

5. The model analyzed by the author was a suggested one. It 

vas used to illustrate the design and indicate the nature of a 
management system. While the terminology employed and the 
illustrations used might suggest that the system would be 
appropriate for business, industry, or institutional use, the 
system could be modified to fit the requirements of an 
educational enterprise* 

6. The model contains a high degree of specificity. 



Systems Model Number 3 

X. William J- Gore, Department of Government, Indiana State 

University, Administrative De cision-Making : — A Heuristic Mode l. 
John Wiley and Sons, Inc., New York, 1964. 

Documentation: American Po litical Science Review, 59-469, 

June, 1965. "This book is a major contribution to organization 
theory . . . Perhaps most significant ... is its implication 
for total political systems ... if his generalizations are 
true for simple organizations ... as well as for the operation 



of larger political systems ... (the book) is tightly 
written ... it is likely to frighten away or lose those who 
most need its message . . .“by Donald Smithburg. 

American Sociological Review , 30:638, August, 1965. "The 
presentation of the model is largely descriptive ... The 
few illus trations given are ... short, very general ... The 
style is uneven . . - the organization leaves more than a little 
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to be desired . . . The last few chapters, dealing with 
decision-making and organizational models . . . are well 
written . . . the book will provide thoughtful reading for 
those working on topics it covers ..." by J. A. Litterer. 

2. Several assumptions are advanced by the author: 

a. Rational systems of action are the organizing mechanisms 
of society. They presuppose an understanding of causes 
and effects, also a stability of goals. 

b. The heuristic process is oriented toward the 
relationship between personal values and ideology. 

1) Through the heuristic process, the private world 

of one individual is linked both to others and to 
the collectively constituted world. 

2} The emotional motivations that energize the formal 
(rational) organizational system are released by 
the heuristic process. 

c. The decision-making process links the conception of 
organization as a rational system with the conception 
of organization as a social system, or as a collective 
heuristic strategy. 

d. Heuristic decisions are mental and vicar:* -*tis ; they 
involve people in thinking about things instead of 
doing them. Decision-making is choosing, not between 
alternative courses of action, but between alternative 
goals . 
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3. Major components and phases of the heuristic model are: 
Phases Components 



a- Perception 



1) Tension articulated as problem 

2) Contingent response 

3) Situation identified as 
indeterminate 



b. Evaluation Sec 



c. Estimation of 
Consequences 



4) Characterization of st imul us 

5) Determining reaction level 

6) Development of orienting set 

1) Reorientation to search for meani n g 
of situation 

2) Search for ideological mea n i n g 
of situation 

3) Definition of organization’s 
stake in situation 

4) Articulation of organization * s 
stake in situation 

5) Consideration of costs of potential 
action 

6) Causal identification of 
alternative responses 

7) Declaration of cost in mounting 
responses 

8) Evaluative set defined in terms 
of stakes and costs 

1) Reorientation to estimation of 
consequences 

2) Reconnaissance 



a) environment 



L ) power centers 

3) Initial formulation of cooperative 
preference structure 

4) Tn-tr-tal attempt to define structure 
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Phases Corpponents 

d. ?*anexiver for 1) reorientation from internal to 
Position environmental interaction 

2) Definition of decision space 

3) review of attitudes toward potential 

response 

4) review concessions that could be 
made to secure sanction 

5) Determination that sufficient 
sanction has been secured 

6) Public pronouncement of proposed 
response 

4. The general model was formulated from conceptions developed 
through four research projects and was augmented by information 
taken from more than two hundred case studies involving more 
than fifty public and private agencies. Illustrative data* 
however, were restricted entirely to a city fire department . 

5. Hhile the ostensible interest of the author is city government, 
the model does not appear to be limited to the operation of 
governmental agendas. Actually * the model appears to be 
appropriate for use in nearly any sort of organized endeavor. 

6. The model is of a middle level of specificity. 

Tfodels for Curriculum Evaluation 

The work of Glass (27) showed four basic models for evaluation which 
he labeled accreditation, Tylerian. management— systems , and sumaative— 
composite. The accreditation and Tylerian models have been utilized 
most often, but the appropriateness of these models for developmental 
efforts is limited. The accreditation model applies arbitrarily arrived— at 
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standards for judging a orovram and the Tvlerian model focuses mainly 
on measuring the attainment of objectives* tending to ignore inputs and 
processes . 

Project consultants felt that either a management-systems or a 
summative— composite model would be most appropriate for this developmental 
project and presented an example of each. 

The management— systems model selected for presentation is by 
Stuff lebeam (68) and the summative— composite model is by Stake (64). 
Subseouent paragraphs present these models and supvest the kinds of data 
that Tvould be included in each. 

Stuff lebeam v odel 

The evaluation model developed by Stufflebeam is rather complex and 
is primarily oriented toward decision— making . Hebert son (53) has 
presented a discussion of its application to the evaluation of vocational 
programs in general. The evaluation of the American Industry Project (46) 
is designed as a management— systems approach very similar to the Stufflebeam 
model. 

Stufflebeam has identified four kinds of evaluation: context, 

input, process, and nroduct. The four first letters of these words have 
been used to form the acronym to name the CXPP Ev alua tion : T odel - 
Figure 2 on the following page is taken from the Stufflebeam paper. The 
material in this figure provides a useful , general description of the 
methods purposes of each of the four kinds of evaluation - 

Whether a context* input, process, or product evaluation is Che 
intention, the logical structure of activities, as suggested by Stufflebe am , 
will be the same. These activities are summarized as follows; 
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L. Focusing the Evaluation 

a. Identify the major level (s) of decision-making to be 
served; e.g. local, state, or national. 

b . For each level of decision-making, project the decision 
situations to be served and describe each one in terms 
of its locus s focus, timing, and composition of 

alee mat ives - 

c. Define criteria for each decision situation by specifying 
variables for measurement and standards for use in the 
judgment of alternatives. 

<3. Define policies within which the evaluation must operate. 

2. Collection of Information 

a. Specify the source of the information to be collected. 

b. Specify the instruments and methods for collecting the 
needed information. 

c. Specify the sampling procedure to be employed. 

d. Specify the conditions and schedule for information 

collection. 

3* Organization of information 

a. Specify a format for the information vhich is to be 

collected. 

b. Specify a means for coding, organizing, storing, and 
retrieving inf onna tion • 

4. Analysis of Information 

a. Specify the analytical procedures to be employed. 

b. Specify a means for performing the analysis. 
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5. Reporting of Information 

a. Pe fir. e the audiences for the evaluation reports. 

b. Specify means for providing information to the 
audiences - 

C. Specify the format for evaluation reports and/or 
reporting sessions - 

d. Schedule the reporting of information. 

6. Administration of the Evaluation 

a. Summarize the evaluation schedule. 

b. Define staff and resource requirements and plans for 

meeting these recuirements . 

c. Specify means for meeting policy requirements for 

conduct of the evaluation. 

d. Evaluate the potential of the evaluation design for 
providing information which is valid, reliable, 
credible, timely, and pervasive. 

e. Specify and schedule means for periodic up-dating of 

the evaluation desipn . 

f . Provide a budget for the total evaluation program. 

Some specification of these activities and suggestions ns to 

methodology and available instrumentation is available to a limited 
extent in the literature; Forthen (86), Wallace and Shavelson (83), 

Burger and Cass (10) » and Caldwell (11) - 

The Stuff lebean model is quite complex in two respects. First, as 
indicated above, it includes many kinds of data. Second, it attempts to 
establish a system whereby the evaluation efforts are coordinated across 




35 



.... ** V *iV 



28 



levels of evaluation. Figure 3 from the ftufflebeam paper illustrates 
a system for coordinating evaluative efforts at the local, state, and 
Federal levels . 

The figure illustrates coordination of evaluation efforts at three 
levels of government. Boxes one, ten, and fifteen could be labeled 
differently * however, and the feedback control loop could be adopted 
as a general plan for local program situations. For example, box 
one might have the label of individual course or learning experience, 
box ten might be labeled local program operations, and box fifteen 
labeled state program operations. Thus, for each course or learning 
experience there would be context, input, process, and product information 
Tliis information would be used to make decisions about the course and 
would also be fed into the overall program evaluation. These data 
from all of the learning experiences would provide the bulk ox the 
information for evaluating the total program, as well as basic 
information for reporting into the state evaluation systems. At the 
top of the loop there would be feedback or information provided from 
the state to the local program in terms of state needs. This information, 
along with the self-evaluation , would be used at the local level to make 
decisions about the local program and the learning experiences in the 
local programs. 

The CIPP Hodel provides a useful way of planning an evaluation 
effort in that it specifies to a great extent the kinds of data that are 
needed in evaluation. It also clarifies the evaluation task by its 
provision for evaluation at different levels, and the fact that at 
each level the data and information needs might differ somewhat, but they 
can and should be complementary. The consultants pointed out that the 
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Development and Evaluation Model presented as an example in the 
project proposal incorporates many of the features of the Stuff lebeam 
Model, especially with respect to the context and input kinds of 

evaluation . 



Stake Kbdel 

The Stake Model is similar in many respects to the stufflebeam 
Model. It is perhaps less complex in appearance in that it does not 
attempt to specify the coordination of evaluation across levels. On 
the other hand, the Stake Model is somewhat more complex in its emphasis 
on gathering standards and judgments as part of the evaluation task. 

Figure 4 is taken from a paper by Stake (64) and is a presentation of 

tlie Stake Model - 

According to the Stake Model, the evaluation task is to first identify 
rhe intents of the program in terms of antecedent conditions, transactions 
to occur in the program, and outcomes. Furthermore, the intended 
contingencies among the antecedents, the transactions, and the outcomes 
are specified. An early task for the evaluation is to determine what 
evidence is available to support the stated contingencies. 

The intents determine much of the data gathering activity of the 
program evaluation. The observations column represents the fact that 
some kind of procedure mill be used to determine whether the intents are 

fulfilled. 

The model as presented suggests that standards are used to comp 
the intents with the observations, and that judgments are made on the 
basis of the standards. The standards are often difficult to establish. 

In some cases, a norm or reference group might be a standard, a standard 
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nay be arbitrarily established by the prof ram staff, or a group of 
experts might set some standards such as in the accreditation typ 
evaluation. A task of the evaluation is to define at least some of 

standards against which the observations are Judged. 

Tbe model is somewhat misleading in that it infers a linear progression 

from intents to observations to standards to Judgments. Certainly some 
of the evaluation will proceed in this manner, but variations will occur. 

For example, it would be important for the evaluator to obtain Judgments 
of various people about the intents even before the program starts. 

Are the objectives of the program the right ones! What is missing from 
the program? These are the kinds of Judgments that are needed earry in 

the program. 

The Stake Model is unique in its emphasis on Judgments as important 
evaluation data. The standards and Judgments columns might well be 
considered as permeating the intents and observations rather than the 
lin ear arrangement It seemingly portrays. 

Content Analysis Technique For 
Occupational Curric ulum Develops 

Oliver (47), in a recent issue of Educationa l Technol o g y, suggested 
that the field of curriculum planning and design maintains the status of 
a relatively crude technology in terms of the lack of forecasting p 
of its products. Oliver believes that some of the lack of power is 
attributable to the preoccupation of the field with scientific rather 
than technological theorising. Technological modes of functioning, he 
suggests, ought to be examined mere closely by the curriculum designer. 

The prerequisite task in improving the rigor of curriculum design, and 
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the burden of the curriculum developer, is the development of a conceptual 
structure for identifying the critically dependent and independent events 

with which curriculum designers must deal. 

Xt is interesting to note that Oliver stresses a three-level 
distinction which has not been used to any great extent in helping tj 
explicate the domain of critically dependent and independent events for 
which the curriculum theorist and designer must be responsible. He 
suggests that these are functionally related categories! educational or 
training demands? contingent institutional, material and/or human 
capabilities ; and the necessary instrumental conditions of learning and/or 
instruction. Oliver suggests that the planner ought to be concerned with 
the development of powerful category systems for organizing demands, 
capabilities, and conditions. 

Gagne’s work (23, 24, 26), when combined with Ec row's Man In A 
World At Work and Fryklund's Analysis Techniq ue for Instruction, present 
a cogent view of the primary elements of what may be described as an 
analysis technique for curriculum development. Occupational analysis 
combined with job analysis and task analysis provide for the 
identification, description, and delineation of available occupations, 
jobs, and the skills and abilities required to perform those jobs. 

As outlined in Figure 5, the occupational analysis process is 
essent ially a linear progression of analytic steps designed to achieve 
the identification and specification of the skills and abilities required 
of an individual to fulfill employment criteria. The principal advantages 
of this approach for vocational-technical curricula may lie in its 
recognition of the need to coordinate planning between the local program 
level, the local administrative level, and the national and state levels. 
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1 ure 5; Principal Steps in a Job. Analysis Approach to Curriculum Development 
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Unfortunately , the feedback channels or information systems that should 
coordinate the operations between these levels are not specified. 

Assuming that different problems and consequently different types 
of information will be required for decisions at the three levels 
supines ted, an evaluation network such as the one suggested by Stufflebeam 
in his Context, Input, Process, Product (CIPP) Model would be especially 
applicable to a curriculum development system that aimed to install this 
tyr>e of process - 

In addition to the absence of feedback channels to control and 
monitor the flow of information for decision making, the model (Figure 5) 
distilled from the works of Ga?ne, Borow, and Fryklund focuses on the 
cognitive and psychomotor skills and abilities required to fulfill a 
particular job or perform a series of related tasks. It does not 
attend, at least explicitly, to the affective aspects of job performance. 
Available as a technique that met wide application in the 1930 s and 
1940’s, and was to see extensive use by the United States Military 
Training Services, occupational analysis or task analysis necessitates 
the rather micro-division of a job into Its knowledge and skill 
components. Moreover, it requires that this analysis yield a statement 
of the skills in the form of observable behaviors. 

In a fashion quite representative of the other models reviewed in 
this report, the occupational analysis sequence does not contain any 
specification of system requirements for implementation or of the 
critical variables that would define such phases of the development 
process as the identification of tasks or the analysis of jobs. A 
rev ealing comparison can be made between the occupational development 
process and an alternative sequence for the development and validation of 
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an instructional system. Such a seouence is depicted in Figure 6. Even 
a brief comparison of the tvo ''models'' shows evident and important 
di ffer cnees - 

The instructional system procedure suggested in Figure 6 is clearly 
more expansive and more inclusive than the occupational development 
process described. It would appear that the job analysis approach, 
for example, could function as a feeder process into the phase called 
Defining Terminal Objectives (2A) shown in Fipure 6. A melding of the 
two programs depicted would seem to present a more realistic picture of 
the evident complexity involved fn the development and implementation of 
a curriculum. Even this combination appears to be Inadequate, and this 
inadequacy is suggested by the absence from both depictions of any 
indication of how the output from either one or both of these processes 
is to be installed and adopted by an instructional system. 

Component Analysis of Curricul um Development and 
Eval^tion Processes: Some Cri teria for Judgment 

Attempts to analyze the curriculum development process into its 
essential components are not without precedent in the literature. *ost 
recently, Dworkin (18) conceptualized curriculum as a social system 
composed of a group of interactive elements of persons, processes, and 
properties organized for the purpose of providing the conditions necessary 
for continuing education. Dworkln’s logical analysis led him to develop 
seven exemplar models which he was able to derive from his general 
systems model of curriculum. Utilizing what Oliver (47) called 
technological theorizing, Dworkin suggested: (1) a model of symbolic 

distance: (2) a model for the expansion of shared meaning; (3) an 
analogue model of the change process in curriculum viewed as a social 
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Figure 08 A Sequence for Development and Validation of an Instructional System 
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system: (4) a Qualitative control of consequences in the curriculum 

system; (5) a model of symbolic orientation; (6> a concern matrix for 
curriculum as a social system; and (7) a model of systemic disorders. 

Wickert (84) investigated the problem of identifying criteria and 
formulating an action model for the process of curriculum development. 
Twenty-four criteria that were judged to be crucial, functional, and 
observable within available curriculum models were identified by 
Wickert. These criteria were in turn judged by curriculum experts and 
then an examination of school systems was conducted to determine whether 
or not personnel within those systems judged the criteria to be 
important. Twenty-three of the twenty-four criteria evidenced a strong 
degree of acceptance in the three school systems Wickert studied. Only 
nine of the twenty-four criteria received both the high rating of th_ 
school systems and the curriculum experts. Although Having some utiliity 
as a description of possible standards for assessing curriculum, 

Wickert *s consideration may bear more importantly on the difficulty of 
obtaining agreement between practitioners and theoreticians on wh~t 



important in a curriculum or a curriculum development process. 

In a manner similar to the procedure followed by the American 
Psychological Association and the American Educational Research Association 
some curriculum theorists have attempted to develop recommendations for 
evaluating curricular and instructional materials. While the goals of 
the professional associations referred to centered on the specification 
of psychological and educational tests, the problems their committees 
faced were rather similar to the problems encountered when curriculum 
theorists attempt to develop comparable indicators. 
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In general, two tyoes of dilemmas must be solved when a curricular 
analysis is performed- The first requires the designation of the critical 
components cf a curriculum, and the second requires the operational 
specification of the standards, and relative importance of standards, 
that will be used to assess the alternative components. For example, 

Tyler and Klein (79) developed sample recommendations on 
curriculum evaluation which they presented at the 1970 American 
Educational Research Association annual convention in Minneapolis (See 
Table 1). The seven sample characteristics: Rationale, Specifications, 

Appropriateness, Effectiveness, Conditions, Practicality, Dissemination, 
are translated into prescriptions for practice. These prescriptions 
are rated as "essential," "very desirable*" or "desirable. 

Admittedly, recommendations of this type can be utilized in severa 
fashions. They may function as reminders to the curriculum developer or 
utilizer. On the other hand, as an input into a model formulation to 
guide the development of curriculum, they would require a detailed 
specification before they could be operationalized. For example, 
recommendation R 1, "The value of the objectives must be substantiated," 
is a curriculum prescription that is more honored in the breech than in 
the observance. For obvious reasons, the attempt of curriculum tech- 
nologists to derrribe empirically the intrinsic value of curricular 
components or of a curriculum development process is a task that does not 

lend itself to scientific investigation. 

Scriven (56, 57) has been a major proponent of the judgment of worth 
of a program or curriculum’s goals and objectives. Scriven has suggested 
that the effort of evaluation to describe the extent to which a set 
of objectives may have been achieved by a particular program is only 
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Recommendations for Evaluating Curriculum 
and Instructional Materials 
(Sample Recommendations)* 



Rationale 

Rl. The value of the objectives must be 



subs t antia ted . 

(Essential) 



R3> 



The basis for 
instructional 



the selection of the content of the curriculum and 
materials mu3t be described. 

( Es s en t ial t 



51. The technical manual should state in detail the objectives. 

(Essential) 

52. Objectives should be specified operationally, i.e., behavioral 
responses of students. 



Appropria teness . _ , . __ . . 

41 . xhe kin d of student for 'whom the curricul u m and instructional 

materials aTe designed should be specified. 

^Essential) 



Effectiveness 



El. Technical manuals should cite sources of available evidence to 
document any claims made about effectiveness and efficiency. 

(Essential) 

E3. Evaluation should be utilized when appropriate in the process of 

instructional development. Also, evaluation should be used when 

materials are completely developed. . 

f Essential 1 



Conditions 

Cl. 



The technical manual must indicate the qualifications of the 
reader which are required in order to use the materials effectively. 

(Essential) 



Practicality 

PI. The technical manual should - 
are required and whether any 
be reused. 



'dicate which instructional materials 
of the instructional materials can 

(Essential) 



F2. 



The technical manual should indicate what may be involved in 



teacher tra ini ng. 



(Desirable) 



Dissemination^^ should be made for continued dissemination of new materials. 

new approaches, and new studies. , , . 

(Very Desirable) 

★Presented by Louise L. Tyler and Frances Klein at the AERA 1970 Annual Meeting, 
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part of the task of evaluation. He has emphasized that the evaluation 
specialist must accept the responsibility for also judging whether the 
objectives or goals were worth pursuing in the first place. Faced with 
the evident realization that this process will not submit to the traditional 
modes of empirical scrutiny. Striven allows that the basis for these 
judgments must lie in the evaluator's sense of ethical knowledge. 

Although prescriptions such as R 1 may present scientific and 
philosophic problems, recommendations such as S 2, "Objectives should be 
specified operationally, i.e. behavioral responses of students," also 
assume that the operationally or behaviorally stated objectxve xs an 
"essential" element of a curriculum. The present controversy in the 
literature on curriculum development suggests that there is at this time 

no strong consensus on this problem. 

Sash (20) developed and field-tested a rating list for use by 
educators who wish to assess curriculum materials and learning packages. 
Hash's categorization of the components of a curriculum utilizes the 
traditional division of the curriculum into four broad categories: 
Objectives, Organization, Methodology, and Evaluation. Each of these 
broad areas is then subdivided and components of the area are rated 
individually and summarily. Table 2 presents that part of the instrument 
that is concerned with curricular objectives. The complete instrument 
is presented in Appendix A. 

Eash (20) has presented some evidence that it is possxble to traxn 
teachers to utilize the scale effectively, if effectiveness is defined 
in terms of inter-rat-r reliability. Here, as elsewhere, one difficulty 
is the implicit assumptions the instrument makes about the relevant 
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TABLE 2 

Objectives Section of Maurice J. Eash*a Instr u m en t 
For The Assessment of Instructional Materials 

(Form IV) 



L OBJECTIVES * T«« 

A* Are there objectives stated for the u?se of material? 

1% General objectives? _ 

2* Instructional objectives? - r ~~ 

3. Are the objectives stated In behavioral terms?* 

♦* If stated in behavioral terms* do the objectives specify* 
a« The type of behavior? 

b% Conditions under vhlch it. will appear? 
c* Level of performance expected? 

5* List examples. of objectives* 



& 



B* 12 there ere no objectives stated for the use o£ the material, 
are the objectives Instead implicit* or readily obvious? 

If yes, please outline below what objectives von believe 
Severn the purpose of the material* 



C* What appears to be the source of the objectives (both stated 
and implicit objectives)? 

1* Are the objectives related to a larger frame of instruction? 

2* Are the objectives specific to a subject skill? __ 

3* Are the objectives related to a breeder behavioral 

pattern* that is to be developed over a period of time? * 

4* What seems to be the emphasis of the objectives: (Check 
as many as appropriate) 

a. Attitudinal* c* Cognitive development skills* ^ ^ _ 

b* Motor skills d* Subject skills __ 

5. Are the objectives* drawn from: (Check as many as appropriate) 

a* A learning approach* ___ c. Demands of subject ^ 

k. Society needs d. Demands and needs of child* ~ 

(Citizenship) * *"* 

D- Quantitative Rating of Objectives 



(DIRECTIONS: Please make an ”x" oa the rating scale below at the 

point which represents your best judgment on the following 
criteria* Please place the ,f x” on a specific poini:* } 




1 

2 



i 1 1 i ! 



Objectives - vague, 
unclear, or missing. 
Those included not 
useful • Fails to 
distinguish between 
general and instr- 
uctional objectives. 
Mixes various types 
of objectives, 
confusing to the 
teacher* 



Average, some of the 
criteria for 
objectives met, some 
missing, at tines 
Inconsistent* 
objectives only 
partially operational 
for the classroom 
teacher* 



The objectives are 
stated clearly and in 
behavioral terms* 

Both general and 
Instructional objectives 
are stated in a 
consistent conceptual 
framework. Excellent, 
one of the best, useful 
for a teacher* 



*A11 terms followed by cn asterisk are defined in the glossary attached to 
Appendix A* i— ^ 
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aspects of a curriculum packape. If the assumptions are acceptable, it 
certainly could be employed in the development process as a set of 
guidelines or a ratine scale. Yet, either use will require some 
training in its terminology. 

What is implicit in the questions 'listed in Table 2 is that the 
objectives of a curriculum or package are, in some sense, a reflection 
of the philosophy or rationale that initiated the program. The purposely 
general level of these kinds of statements makes them far more difficult 
to analyze than a set of objectives with a high degree of specificity. 
■Whether the analysis is performed on the objectives or the rationale, 
one would expect some degree of congruence to exist between the two. 

Whether or not it is fair to dichotomise responses to questions such 
as (1.A.3) "Are objectives stated in behavioral terms? '* may certainly 
be debated t but the more pertinent questions might be: 

1. Hot -7 will the system incorporate the development and statement 
of objectives? 

2. What alternative routes are available for achieving the 
statement of objectives? 

3. What s kil ls and materials will be needed for this specification 
to occur? 

4. How much time and money will be required for the process to 
be completed? 

5. is there a point of critical trade-off between resources 
required to obtain such a specification and the benefits that 
accrue to the student? 

Although Eash * s device was designed as an aid in inpt.it evaluation, 
it is displayed here because it clearly implies one expert’s judgments 
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on what the critical components of a curriculum packape are. Translated 
into a list of goals to be achieved during a development process, what 
is clearly absent is the weight that should be attached to each of the 
four broad categories (See Appendix A) , and to the individual components 
of each of the categories themselves. 

Insofar as curriculum evaluation can be seen as synonymous with 
curriculum development, some attempt will now be made to s umma rize the 
pro to typic models of curriculum evaluation . Drawing primarily from 
Stake (66), we will categorize the models as to their: Key Eophasis, 

Purpose, Key Activities, Key Viewpoint Used to Delimit Study, Outside 
Experts Needed, Expected Teaching Staff Involvement, Risks, and Payoff. 
These eight characteristics are applied to five prototypic evaluation 
procedures; namely, those suggested by; Tyler (81), the accreditation 
movement (4'>), Stake (64), Stufflebeam (69), and Tnba (78). Judgments of th 
correspondence between these evaluation procedures and the eight 
characteristics are summarized in Table 3. 

In addition to summarizing what may be the trade-offs necessitated 
by the selection of one curriculum e valua tion process or another. Stake r s 
analysis implies what each of these evaluation processes holds to be 
important about curriculum evaluation. Insofar as that is accomplished 
("Risks” and "Payoff”), his suggestions may provide an additional basis 
for the designation of the critical elements of curriculum development. 

One could, and possibly should, differ with any one of the entries 
in the cells of Table 3. It is no easy task to isolate a "'Key Emphasis” 
or "Purpose," whether these be in curriculum development or evaluation. 

One unfortunate application of an analysis uch as the one depicted in 






3 

m 

a 



CO CJ 

3 a 

3 8 
H *• 



*kJ c 

^ w 2 
Ai ««o 
O C > 

O V» H 

ax o 

X o > 

Ui m a 



•o *> *o 
-h u o 
« 9 -c* 
u a «» 
3 X 0 

o m as 



c m 

^ 3 

o o « 

g-fr* CO 

> o s 

3 -H 

>* »H 

2 & 



M | 



*» 



5S 4b 

a? 



u 

«< 



X ** m * 
t> {3 9 
h 7 b 
«*»i oc 

2SE 

< « a 



§1 



• o w 9 

h O O u 

5 *5 & 2 

O ««4 0< 



-4 0 9 
a > w 
3 -4 Q 



a *3 > 

5°S 



« u w 
« fa# c 9 

> —4 £r •— t 
«M V « 
*i -H U M 
O Cl 3 Cl 
O V « « 

na< a 

X V 3 ^ CO 



5 $ 

H Ml 
3 *H |4 
U > « 
■*« fa* jc 
Vl 9 U 

v« a< 

3 3 9 
U W H 



Hs 

ox . __ . . 

ass S 3 



® o 4 b 

_ M S.^ 

U 4 • H 



O fa* 



3 «M 
9 «•-» 

X “* 



O 3 m 

. O -4 *b 

X 4b «0 9 3 

W « M > O 



w 

& 

3 

A« 



Cl 9 

42. -3 

X ** • 
O CO Cl 

o 

XV o 

^4 V« «l 
I -*4 3 9 
I o « a 
j « 9 « 

1 J 5 * 2 ° 

w 6 o 



® > 
9 -4 
I- V 
CO 9 
O « 

Vi «n 



s! 



V CO 

S CI >4 
*T 3 «Q 

O ** O 
H a 



fl 

O 



o 

ERIC 



O O O 
a-H b 
X *4 3 
*4 VIS 
0 3 9 
*> H V 
0 9 0 
b > w 
6* W P* 



« > 

O -I-C 

3 41 
b o 



.53 



9 a 

H O 

£*3 

«-4 

X 3 H 
a — t «y 
«w ® m 

9 > O 



• § 

O <%"• 



II 

Q --4 

O 9 



9 1 9 

9 9 9 

7 ? 42 «*• . 

5 u x« 

C X <U 3 

9 

v x e a 3 

C 4 « 3 b v 
0 3^30 
2 & 9 4 -» o a 



g-t 

O b • 
b « «M b 

«x c o 
• « 
9 9 a 9 
o -O b 

u H ■< *4 



9 O 

b 

•O 0 9 

O 9 

b 9 
a*b 9 

« 2S 

3 ~2 

0 0 9 

« 3 

3 g. 2 . 



C O 

9 

4 b 9 

C 9 

O M 
ci 3 a 
^ m o 

» 9 <vf 
9 Cl 4 b 
WOO 
> b 3 



pec 

How 



!K 

*T 3 

3 



.«.! 
It U 44 M 
3 H O O 
*4 9 %4 4 t 
O O H O 
W M \m tt 

"Sgl 

^ 3 <4 * 

a u. 

o <o o* 

bw C e 
« O %«3 



e& 9 

4 b Cl 

a u 
3 w e 

S w b 
cm o 
<** ++ a e 
44 o o 9 
« > O -* 



o <3 



O Qi 
H O 



I 5 - 

V w 4 
w ^ X 
b M 3 

3 H C O 

9 9 

9 «M 



9 > 

& 9 



9 — • O 

p m js 
N W o 
3 W X 
0 0 9 
m u a 



® 9 

O >4 
9 



72 ** 

3 -4 



js 9 5 

S S 3 a 

9 9 4b 
9 > a 

> y v* o 
MW 03 
• W3 9 

St §s 
s 

9 * o 

«-* 9 a 

9 C 9 

o. o .«• 

W C 
O • w 
W W b 

4b o a 
d « o 

<30 



3 

O 9 



cw C 
* O -< 



b 
I O 

■d 4 b 



SS b 

52 



3 1 - 

i-< « 9 

23 3 3 

a ii 

O fa MO 

355 3 . 



5 S- 

O MM 
#• 4-4 0 ® 
W 0 9 
*■**•9 3 
U X b H 
b b O 4 

<w a> 



5 * 

d w 
9 9 



b b O 

o o b 
i-W cu c 

p x o 
H • O 



9 
- Cl 
XMb 
Cl 9 4 b 
b C 9 
9 oaob >* 

« vl W H 

9 9 0 9 

2 2 £5 



b 

- 4 A O 

*4 x 

9 Cl 
W H 
M 9 
O 9 



oS 



9 X 0 
« 41 o © 
9 C *-* 

> 5 ~ 5 

•b 9 O O* 
o OX 

> C 4 b 

0 9 9 

o W i X 
43 O b 
W 3 c 4 
£9 9 x 2 VO 



• ex 

X 4 ) b 
4b b 

0 9 8 

b<M e) 3 
lib «h 
0^4 0 

a 3 9 o 
9 W <H 

b « a b 
X O b 
0 9 9 3 
H > 0.0 



Si 



_ t* w 

M 3 b o 

C b « b 
«-# 9 9 4 b 

8 c 

o • - o 

Cl 9 9 9 
Cb O C I 
3 > O X 

ri M b 
X« b w 
4 m 9 9 «W 
W C Cl 4 
b b b 9 
COHO* 

o b a 
3 w 0 a 



19 1 

-w w 

b H b 
9 0 9 
a fa > 

X 4b 

9 C Cl 
O W 
9 9 4b 

« a 

w w E 
0 9 9 



2 c 
£ 2 



X « 
9 9 



4» 

4b < 

Cl 



0 

w 

Xtf 



CO 

t 



b ^ 



- ?f 

H W W fl 

W 4 3 Q 

y c c w 
d o w 4 
«M W 41 <H 
4 b C o 
0 9 0 0 
H fa 09 



I 

o 



— t o 

Cb M 
6 C CS 
W W O 4 
4 faWX 
3 b b 
5*0 O 
« C C > 
9 9 0 
• w b 

Cl O 
O 3 X X 
H X « 3 



O 

2 X 

4b 

—4 

9 

b 

O 

> 

•*4 

C 



3 

4b 

3 



O «r 4 
O b o 

•com 
o o o 
«J < x 



2 S 2 



u 

c 

9 
9 C 
b «<n 
VI 44 H 

,» g-S 
Si-3 2 



3S 

o ^ 



S 

9 
9 

3 * 

<3£«2 



0 9 9 

3 fab -4 



3 J 2 

4 b Z 

co 

e £ 

2 S 



53 



8 

O 



h riA 
9 9 3 —4 
m **4 — <u 
«4 O (5 9 
W O > Q 
SSS CO ru s: 



45 



o 

2 



C» 

Oh 



b 

o 

-o 

O 

4b 

o 

o 



s 



3 

—4 

£ 



o 

o 



u 

o> 

>v 



C 4 C 
Oh O 
•tf O 



o 

•w 

>v 
• 9 
•8 3 

U 4b 

fa CO 

5 h 

xTiSJ 

9 Z 
fat 

« • 



o 0 

I s 

O a 

• «S o 

4 —h 9 9 
C > 4-1 
0 4 0 

MOW 



3 8 . 



\ JZ 

3-4 0 
rv 9 3 b 
O III U M 
Os W « 
-4 44 fa* « 
O m m 
• _ 3 c 2 
W 4 o 
4 oo 9 
fa 4 9 40 > 
aw w 

<H C b 
O O C 
-O -4 fa* 
0^90 



e ° 



«-v 

I 9 fa# Q 
O b « O 
<m c a 

«*V V 3 • 

o o a 

-* 9 

o a fa b 
g c 



0 

O 



*— I 0 
9 3 
> -4 
9 9 



O U 



>90 

« J 



. 9 
jC 



9 

< >* 

• b 

«> >v 9 

C JO 4b 

w c 

bC rn a 
9 9 8 
& X o 
« <0 -4 



0-4 VO 
W X c o 

(D 3 v« Oh 

* ^w a. -v 
c a oe 

O 09 C- 

'-i m «o «-* c 

4b C Cl 

; fap o o 

3 O H w w 
H 9 b O 

3 • ci c 

> bb\ c « 

9 C O 3 U 

g d W X 

e 

« C w o c 
C 9 e > 9 
O 4b w CO 
-r* JC 4b 
4b oc* 
d W hO 

U *4 Ov M 60 
3 C -4 o 9 

*0 9 Cl *W 



& 3 





> 

c w 


9 — — 4 

«» o m o 

«M 0 44 C O 


— 


o 


w 


-4 


O JC 0 


O 


4 b O 


tc o 9 


eC 


c o 


9 0 O 4 b 


w 


O JC 


Cl w - o fij 


o 


E CI 
« CO 


0 4 b as -4 4 b 
0 9 3 COCO 


« 


LI 


c 3 XV O 


9 


9 >v 


O H £ w <5 


x£ 


4 b b 


-L# 0 3 0 O 


9 


9 9 


C >W b « 


4 b 


*o 


3 HI O b ^ 


CO 


—4 C 
9 O 


o o o c* 
« • 0 


• 


fa y • 


•J • M <9 


w 


0 9 0 
c to 


3 X C b 

XH Ufa fa, 


4 b 


9 • 


H d w x 


b 


Ufa o 


■*"■9 9 0 


0 


o 


• C h* 9 «« 


X» 

B 


3 X c 


« d c o w 
Q > H 



O 4b ••4 



* 

3 9 
9 
fa C 
0 O 
O- *- 
0 9 
b U 4 
CL O 
4 C -- 



X 3 b b - 0 
cl 44 eco 6=3 so 
w w c X 3 3 < 

<3 v« c a 9 w d 
KWXI3 X b M 

0 » *j as . 

* c f! »r-f b a 

fa 0 3 3 w « *2 

0—4 JC h *4 

*w 4b 0 3 x 

X 3 u u ta 

H Z to CO fat - 

1 r>^ an 



46 



Table 3 would be to allow it to constrain the reader to choose either 
one or another of the suggested prototypes . What is more clearly 
required is a melding of the elements of several in order to satisfy the 
ratio nal e of a particular program or project. 



Observations on Curricular Differences 
One way to begin to separate alternative curriculum development 
processes is to examine some notions or questions relating to learning 
and the assumptions that may underlie these processes. Grobman (32) has 
identified the following as critical Questions that should be considered 

during a curriculum development project: 

1. How do students learn most effectively? Do all students learn 

best in the same way? 

2. Do students learn because they are rewarded and/or punished; 
that is, what is the role of motivation in learning and how is 
mo tivation fostered ? 

3. Can one train the mind or the faculties of the mind, with the 
result that general competence in all intellectual areas will 
be enhanced? 

4. Will knowledge of the past — particularly of the classics of 
the past — prepare students to deal with the problems of the 




5. 



6 . 



future? 

Pill training -ir> one area be transferred automatically to other 
areas? Can such transfer be expected to occur only with 
specific preparation for transfer? Or can transfer not be 
expected under any conditions? j 

What is the role of insight in learning? What kinds of Insight 

j 

are important:? How are these developed? 
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7. What levels or types of cognitive skills enhance retention of 
learning? What levels of cc&nitive learning increase ability 
to use present learning in later high— cognitive level tasks? 

8. Is learning most effective when it proceeds from the known to 
the unknown or from the unknown to tl;e known? 

9. Should learning go from the abstract to the concrete, or from 
the concrete to the abstract? 

10. Should learning be sequential, and pyramided on previous 
knowledge? 

11. Can complex cognitive abilities be mastered during the child f s 
first school year? Or can they be mastered only when taught 
initially during the child’s first school year? 

12. To what extent are children born unequal in terms of 
intellectual potential? Is there a great variation in the 
innate, intellectual ability of students? E ring pre-natal or 
post-natal injury, can all children be brought up to an effective 
functional level? 

Crobman points out that questions such as No. 3 have been dismissed 
by the majority of learning theorists as untenable. This dismissal is 
anchored in the aversion of contemporary learning theorists to an earlier 
school of thought that has come to be called "mental discipline.” On the 
other hand, most of the questions remain rather open-ended, with more or 
less empirical evidence available on them . Consequently, if changes 
in behavior or the capacity of students are to be designated as the 
important outcomes of transactions that occur tinder the aegis of a 
curriculum or its instructional component, it would seem to follow that a 
development process will have to attend to several of these questions. 
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The onus of Grobroan*s questions falls on the behavior theorist. 
Without reviewing the gap between research end practice, it would seem 
that the output from a curriculum development and evaluation model should 
reflect relationships between curriculum and learning theory. Whether it 
is presently possible or advisable to draw meaningful differences between 
alternative instructional methodologies (See Appendix A, Section III) may 
be debated. Unfortunately, scientific inquiry into this area, as 
summarized by Rosenshine (54), suggests that the profession evidences a 
"significant lack of knowledge" (p. 661) about the critical variables that 
relate instructional methodology and student learning. 

Grobnan goes on to develop a comparable list of problenatic 
questions from the realm of educational philosophy and value judgment 
and suggests that these, too, are relevant concerns for the curriculum 
developer. Her summary of the problems involved in ascribing a particular 
theory of learning or philosophy of education to curriculum has relevance 
to the development of a systems model for curriculum development and 
evaluation . 

The fact that theories of learning, philosophy of education, 
and value judgments have not been identified by developmental 
projects as guidelines, does not mean that they are not part 
of the assumptions underlying the curriculum. Where these 
assumptions are not identified, it is harder to evaluate 
them, to check them for consistency, to be sure they are 
mutually compatible and supportive, and to insure that they 
represent a valid reflection of what the project wants to 
do. (p. 110) 

It appears that the development of an operational model to guide 
local educators in their attempts to develop curriculum will certainly 
have to attend to the points raised by Grobman. 

Within this broad frame of reference, it may now be possible to make 
some useful generalizations across alternative curriculum designs. 



O 

ERIC 



56 



49 



Initially, it appears that one critical dimension alone which alternative 
approaches to curriculum development may be distinguished is their approach 
to the selection and specification of the content which is to form the 
core of the instructional experience. For example, the curriculum 
projects of the fifties and sixties (i.e. Biological Sciences Curriculum 
Study) may be characterized as curriculum movements that were generated 
in an attempt to revivify the relationship of the disciplines and the 
K-12 curriculum. In an attempt to do this, these projects were typified 
by the utilization of content experts from the humanities and sciences 
who were called upon to testify to the contemporary validity and relevance 
of the curricular offering of the schools and where necessary, to change 

or augment it. 

Cn the other hand, the broad perspective that is possibly provided 
by one of the Tylerian models, Taba's social studies model for example, 
would seem to approach the problem from a different point of view. 

Instead of beginning with testimony from content experts, Taba's 
approach to curriculum is anchored in a survey of the society in a 
attempt to describe its needs. lu«s description of needs is then 
translated into a series of objectives which become the aims of the 
curriculum. It seems that the origins of the content-centered curriculum 
projects and those models that might variously fall under the Tylerian 
approach are different in content origin; the first beginning in the 
disciplines, the second beginning in the society at large. 

In a si mila r fashion, the procedures that have been variously 
collected under the banner of occupational analysis appear to represent 
an approach that is more comparable to the Taba model than it is to the 
content-centered curriculum. Where systems models fit when compared to 
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. either one of the preceding approaches is difficult to surest. What 
seems to be clear is that systematic approaches to curricula- development 
are characterized not so much by definitions of vhat is important to be 
learned or hoe learning is going to occur, so much as they are by 
attempts to order efficiently that process over time via the designation 
of critical decisions and the factors that may be predicted as influencing 

these decisions. 

In addition to the origins of content as a point of viea 
which alternative approaches to curriculum development may he identified, 
a dimension called relative complexity may differentiate the models 
discussed. For example, as was pointed out in the section on reviewing 
curriculum models, one characteristic that typifies the product 
development model is its antecedent requirements of high degrees of 
technical coherence in facilities and personnel designed to conduct that 
effort. On the other hand, the more 'traditional approach suggested by 
Tyler has had wide use by schools that have attempted to modify their 
cu: ricuiar strains , although its application may have occurred a 
rather simplistic level- 

What seems to happen in the on-going project at tne school level - 
that neither one modal nor another is chosen exclusively. Rather, it 
would appear that particular elements from one or several models are 
intertwined into a sometimes rather nebulous network of operations. Fo- 
example, it is not unlikely that some type of needs assessment study, 
survey of the local community, or search of the professional literature 
within a particular domain may well preface an organized attempt to 
develop or initiate a curriculum strain. Aspects typical of several 
models, such as the specification of instructional outcomes in the form 
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of observable behavior and some attempt to indicate the required 
frequency of response in these categories. may also be included. The 
differentiation seers to occur mhen the rather minute specifications of 
Popham and Baker, for example, that deal vith the technical requirements 
for the specification and measurement of criterion performance, are 
described • 

A motif that seems to have run through much of uhat has been sard 
to this point is that dimension of curriculum develop.-— nt tha- 
freouently described as the implementation of the curriculum into an 
instructional system. In addition to models describing curriculum 
development, there does exist an the literature a s-r^ s 
systematic change or innovation. For example, the work of Clark and 
Cuba (14) is frequently cited as a research and development model tnat 
can be used to guide the Implementation of a curriculum into an 
instructional system. This type of approach to change has been 
criticized of late by House, Kerins, and Steele 06) in their review of 
its effectiveness as e rationale underlying the Illinois Gifted Program. 
Tet, one cannot dismiss the importance of developing a system that does 
attend to the problems involved in implementing the curriculum in a 

human network. 

, - m /i pi has suggested that curricular 

Speaking to this problem, Eash (lw “ as 

, . „„ ni fpration of classroom practice, 

change, when that change is described as al 

. roeh identified them as: 

may be ascribed to three alternative models. Eash i - 

owhe co-action model: and (3) the displacement 
(1) the authority model; { 2 ) the co acci 

model. After a discussion of each, it is his conclusion that the co-active 
approach to curriculum change, an approach characterized by an early 
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and meaningful cooperation between researchers and developers, is the 
only approach that has legitimacy and possible utility for effecting the 
alteration of educational programs. 



O 

ERIC 



60 



CHAPTER III 



CONCLUSIONS AND IMPLICATIONS 

The investigation of models for curriculum development and 
evaluation in the literature rapidly revealed that the concept of a 
model, i.e. what it is and what it is supposed to do, has little real 
utility, except as a piece of appropriate jargon. It was hard to avoid 
the conclusion that there are, presently, few if any actual models of 
curriculum development. This may in part reflect the eouaily obscure 
role that has been held by theories of curriculum. 

Difficulty was experienced in comparing models because each of the 
following variables played a part in determining the overall advantage 
or disadvantage of a model: 

1. How ready is the institution for change? 

2. What expertise can be expected to play a role in the development 
process? 

3. What is the nature of the desired change? 

A . Where are the pressures for change originating? 

5. Who will initiate the change in curriculum? How will the new 
curriculum be institutionalized? 

The following is a list of conclusions and implications for the 
Illinois Occupational Curriculum Project, drawn from the review of the 
literature: 

1. The alternative strategies for curriculum development available 
in the literature may be broadly divided into Tylerian Models, 
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Systems Models, and Product rove 1 opr on t Models. The application 
of these models at the local level is rarely a pure adoption 
of one over another, but is rather a combinatorial process. 

The state of the art in curriculum development presently 
evidences little forecasting power as a consequence of the absence 
of either sound scientific or technological theorizing. The 
model for curriculum development and evaluation being preparea 
by IOCT should address itself to this fact. 

The contingencies that compose a curriculum development process 
may be broadly categorized as, (1) educational or training 
demands; (2) contingent institutional* material, and/or human 
capability demands; and (3) demands o£ the necessary instrumental 
conditions for learning and/or instruction. These contingencies 
should be reflected in the IOCP mo dei. 

As an analytic attempt to describe the components of a task 
performance, the strategies of occupational analysis present 
a tried procedure and should be adopted for use in the program 
development section of the IOCP model. 

Curriculum models are not noted for their evaluative component. 
The occupational analysis approach should be supplemented with 
a systematic evaluation process in the IOCP model, not unlike 
the CIPP Model developed by Stuff lebeam. 

The development of any evaluative system from among the suggested 
prototypes should provide the user with flexibility and the 
opportunity to attend to alternative evaluative procedures. 

Since the system developed will be utilized by a variety of 
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institutions, an effort should be made to allow for the 
planning and implementing of an evaluation prog-am fitted to 
the needs of the particular program or institution. 

7. The IOC? model should allow for the assessment of curricular 

objectives and goals in such a fashion that the value is judged. 
Father than just stating "Are objectives stated in behavioral 
terms?" the more pertinent questions might be: ( 1 > How will 

the system incorporate the development and statement of 
objectives? ( 2 ) What alternative routes are available for 
achieving the statement of objectives? ( 3 ) What shills and 
materials will be needed for this specification to occur? 

(4) How much time and money will be required for the proc_ss «-0 
be completed? ( 5 ) Is there a point of critical trade-off 
between resources required to obtain such a specification and 

the benefits that accrue to the student? 
s. Regardless of the specific nature of the model developed, there 
is a tradition chat suggests it will have to deal with four 
primary aspects of the curriculum that ia, <1> the development 
and statement of objectives; <2> the organization of objectives 
and content; (3) the presentation of alternative methodologies 
for instruction; and (4) the internal and external evaluation of 
both the process and product components of the curriculum 
9, The IOCP model should attend to both the discipline and the 

society at large in determining content for learning experiences. 
Occupational analysis is s technique for determining technique 
that may more closely relate to the discipline approach. 
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10. The systems approach to curriculum development attempts to order 
o?f±ci*nely, over time, the critical decisions and factors that 
may be considered as necessary and most Important. These factors 
considered could renresent any philosophy of curriculum 
development. Therefore, the development of the IOCP model 
should attempt to consider the critical aspects of various 
theories and philosophies of curriculum development. 

11 . i n order for the IOCP model to be practical as a guide to 
assist administrators in making curriculum decisions, it must 
contain sufficient detail to suggest activities at an operational 
level and yet avoid complexity and detail that would require 
excessive expertise and time. 

12. The IOCP model should attend to the human problems involved 
in implementation. 
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An Instrument for the Assessment cf Instructional 
Materials (Form IV) 



I. Objectives Yes^ No 

A. Are «-here objectives stated for the use of the material? 

1. General objectives? 

2. Instructional objectives? 

3. Are the objectives stated in behavioral terms?* 

If stated in behavioral terms, do the objectives specify: 

a. the type of behavior? 

b. conditions under which it will appear? 

c. level of performance expected? 

5. List examples of objectives: 



B. If there are no objectives stated for the use of the material, 

are the objectives instead implicit* or readily obvious ? 

If yes, please outline below what objectives you believe 
govern the purpose of the material. 



C. What appears to be the source of the objectives (both stated 
and implicit objectives)? 

1. Are the objectives related to a larger frame of 

instruction? 

2. Are the objectives specific to a subject skill? 

3. Are the objectives related to a broader behavioral 

pattern* that is to be developed over a period of time? 

4. What seems to be the emphasis of the objectives: (Check 
as many as appropriate) 

a. Attitudinal* c. Cognitive development skills* 

b. Motor skills d. Subject skills 

5. Are the objectives drawn from: (Check as many as appropriate) 

•* 

of subject 
and needs of child* 

(Citizenship) 



a. A learning approach* 

b . Society needs 



c . Demands 

d . Demands 
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D. Quantitative Rating cf Objects 

(DIRECTIONS: Please make an s 'x" on the rating scale below at the point 

'..aich represents your best judgment on the following 
criteria. Please place the : V* on a specific point.) 




Objectives — vague 
unclear or missing . 
Those included not 
useful . Fails to 
distinguish 
between general and 
ins true tional 
objectives; mixes 
various types of 
ob j ec tives , 
confusing to the 
teacher . 



Average, some of 
the criteria for 
objectives met, 
some missing, at 
times inconsistent, 
objectives only 
partially 
operational for 
the classroom 
teacher . 



The objectives are 
stated clearly and 
in behavioral terms . 
Both general and 
instructional 
objectives are stated 
in a consistent 
conceptual framework. 
Excellent, one of the 
best, useful for a 
teacher . 



II. Organization of the Material (Scope and Sequence) 



Yes No 



A. Has a task analysis* been made of the material and some 

relationship specified between the tasks? 

B. If a task analysis has been made, what basis was used to 

organize the materials : (Check as many as appropriate) 

1. Errorless discrimination* 4. General to specific 

2. Simple to complex 5. Logical order 

3. Figure-ground* 6. Chronology 

C. If no indication of a task a n alysis has been made, what assumptions 
do you believe the authors have made concerning the organization 
of the instructional sequence of the material? 



D. Is there a basis for the scope of the material included in the 

instructional package? 

1 . If there is a basis, is it: 

a. related to a subject area? 

b. to a motor skill development? 

c. to a cognitive skill area? 

d. to an affective response system*? 

e. other (please specify) 
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Yes No 

2. Has the scope been subjected to analysis for- 

a. appropriateness to students? — 

b. relationship to other material? — 



E. Is there a recommended sequence? 

1 . Vhat is the basis of the recommended sequence . 

many as appropriate) 



(Check as 



a. inter-relationships of a subject* 

b. positive reinforcement and programmed sequence 

c. open ended development of a generalization* 

d. advanced organizer (cognitive)* 

e. other (please specify) 



F. Briefly outline the scope and sequence: 



G. Quantitative Eating of Organization of the Materials (Scope and Sequence) 

(DIRECTIONS: Please make an "x" on the rating scale below at the point 

which represents your best judgment on the following 
criteria. Please place the "x” on a specific point.) 




Sequence illogical or 
unstated, teacher is 
left to puzzle it out. 
Does not appear to 
have subjected material 
to any analysis to 
build an instructional 
design. Scope is 
uncertain, seems to 
contradict sequence. 
Little help un- 
intentionally to 
teacher or children 
in organizing 
material . 



Average in 
organization . Some 
help but teacher 
must supply much of 
organizational 
sequence. Scope some- 
what limited, may be 
too narrow (or 
broad) . Sequence is 
net detailed enough 
and may not have been 
tested with a range 
of children. 



Excellent organization 
of scope and sequence. 
Conceptually developed 
based on a consistent 
theory ; task analysis 
or other appropriate 
investigation has been 
done. Tested for 
appropriateness of ; 
recommended sequence . 



III. tfethodology 

A. Does the author(s) and/or material suggest any methodological 
approach? 
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Yes No 



B. Is the methodological approach, if suggested, specific to the 

mode of transaction? 

1. Does the mode of transaction*: (Check as many as appropriate) 

a. rely upon teacher-centric method* (largely teacher 

directing)?* 

b. rely upon pupil-centric method* (largely self— 

directing)?* 

c. require active participation hy the students? _____ 

d. passive participation by the students? 

e. combination of active and passive participation by 

the students? 

f. direct students * attention to method of learning as 

well as the learning product? 

g. provide for variation among students — uses several 

approaches to method? 

C- Does the mothodology suggested require extensive preparation 

by the teacher? 

1. How much deviation is permitted in methodology? 

Much Some Little 

2. Does the methodology require unusual skills obtained 

through specific training? 

3. Is there any statement on how methodology was tested; 

any experimental evidence? 

4. If you have tried the recommended methodology, how successful 
did it seem for your students? 

Most succeeded Approx, ha.1 f succeeded Few succeeded 

a. Please provide a brief description of the students who were 
successful and those who were not successful. 



b. What variations on recommended methodology have you used? 



D. In a brief statement, describe the recommended methodology. 
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E. Quantitative Rating of Methodology 

(DIRECTIONS: Please make an "x" on the rating scale below at the point 

which represents your best j udgnent on the following 
criteria. Please place the "x" on a specific point.) 



1 1 


1 1 


1 i 


[ 




r 

L 






l 



1 2 3 A 5 6 7 



Very little help is 
given on methodology, 
or methodology is too 
abstract and complex 
for most students and 
teachers.. Methodology 
appears to be 
unrelated to content 
and an afterthought 
in the learning 
package. Too active 
or passive for most 
students. Teacher 
required to 
participate fully 
with too many 
students at every 
step. Does not 
have appropriate 
methodology for 
variety of learning 
ability among 
students. 



Gives help to the 
teacher, but would 
like more - Some 
students would be 
able to cope with 
suggested method- 
ology, but others 
not. Does not appear 
to have been widely 
field tested. Teacher 
has to work out 
variety for students 
with special learning 
difficulties. 



Uses a variety of 
modes In the 
transactions- Does 
not chain a teacher 
to a mode without 
reason, but provides 
assistance for 
different abilities. 
Describes the field 
test of the method- 
ology- Teachers will 
find methodology 
easy to use and 
believe students will 
respond. Methodology 
Is part of goals of 
instruction and not 
just vehicle for 
content - 



IV. Evaluation 



Yes No 



A- Are there recommended evaluation procedures for teachers and 
students in the instructional package? 

1. What do the evaluation procedures emphasize? (Check as 
many as appropriate) 



a. Cognitive skills c. Psychomotor skills* 

b . Subject skills d. Affective responses* 

2. Are the evaluation procedures compatible with the 
objectives? 
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Yes Ko 

3. Are evaluation procedures developed for several different 
levels: (Check as many as appropriate) 

a. issheaiate feedback evaluation for the pupil 

b. evaluation for a variety of the areas in SI above, 

and over a period of tine 

c. immediate feedback evaluation for the teacher 

d. evaluation on a norm referent* 

e. evaluation on a criterion referent* 

B. Are the evaluation procedures contained in the package? 

C. Does the evaluation give attention to both product and 

process learning? — 

P. Xs there information on how evaluation procedures were tested 

and developed? — 

E. Briefly state what evaluation procedures are included, if 
possible, and give examples: 



F. Quantitative Rating of Evaluation 

(DIRECTIONS: Please make an "x” on the rating scale below at the point 

which represents your best Judgment on the following 
criteria. Place the 'x" on a specific point.) 



12 3 

Haphazard in approach. 
Product and process 
learnings either 
entirely neglected or 
confused. Lists items , 
but poorly constructed, 
no evidence of testing 
of evaluation approach. 
Students receive no 
assistance through 
feedback. Fails to 
recognize and examine 
different types of 
learning where 
appropriate . 



I I 

4 5 

Sene examples given, 
range of evaluation 
limited. Samples given 
but limited and 
sketchy. Teacher finds 
useful that which is 
given, but needs more 
examples . Evaluation 
is limited to product 
or process- Unsure on 
whether evaluation has 
ever been tested, but 
seems logical though 
limited in types of 
learning examples - 



— t — i 

Many suggestions and 
helps in evaluation 
for the teacher. Has 
criterion reference 
procedures where 
appropriate . Student 
obtains assistance in 
learning through 
feedback evaluation. 
Gives attention to 
several kinds of 
learning , consistent 
with objectives o£ 
learning package. 



O 

ERIC 



78 



71 



COMMENT 



A. Draw up an overall statement of the strengths and weaknesses of the material 
as an Instructional package. Prepare your statement as if it were to be 
addressed to your fellow classroom teachers who are going to use it to m ake 
a decision on these instructional materials. 



B. Quantitative Rating Overall Assessment of Material 

(DIRECTIONS: Please place an "x" on the point in the rating scale which 

best represents your overall judgment of these materials. 
Place the "x” on the specific point.) 



I 1 1 1 1 1 1 



Poorly designed, 
conceptually weak and 
inconsistent or 
haphazard design. Does 
not appear to have 
been field tested: 
Inaccurate assumptions 
about children who 
will be using 
material. Overpriced, 
underdeveloped , a 
bad bargain. 



Has strengths and 
weaknesses, but most 
teachers would find 
satisfactory. On the 
balance comes out 
about average, would 
need considerable 
supplementary effort 
by teacher. A 
compromise or price 
and availability. 



Excellent, one of the 
best by comparison 
with other available 
material. Theoretically 
strong and carefully 
field tested. Shows 
consistent Instructional 
design. Would recommend 
highly, well worth the 
price. 
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A GLOSSARY OF TERMS 
USED IN THIS INSTRUMENT 



5. 



*Obiectives stated in behavioral terms - a work picture of the type of 
behivior product which one might expect when the objective is achieved. 
ObVaUi-es stated in behavioral terms will usually name the behavior, state 
thl conditions under which it will appear, and the level of performance 
expected e.g. the child will be able to spell (type of behavior), in 
fo^rid iSfor^al writing (condition under which it will appear), 93% 
of the words in his written work (level of performance) - 

^Implicit objectives - an examination of the content will permit the 
reader to readily identify the objectives that the student 
„ _ nrrmlish even if the producer has not stated them. If a •- P 

giv^Tthe sequential steps in solving arithmetic problems using long 
livision, one would assume the implicit objective to be to teac 
student the process of long division. 

;s:fioioS°ihat^^ 

SSlST; broade^behavioral^pattern S^h^ns^ge^r a ^ of 

material miy be designed to contribute to a ls 

Spired to^ single^^picifiS behavior there should be some relationship 
to a broader behavioral pattern. 

*Attitudinal objectives - objectives that are designed “ develop 
■reelings and predispositions to act in accordance with internalxz 

sr 

:*"SSS^ evaluating the n^s hy 

investigating the source of reports. 

^Cognitive development skills - objectives which have cognitive 
development skills (thinking) as a basis will usually ^asise tn inkin g 
processes as their focus, such as understanding, "f* ^ 

chaining and evaluating as opposed to emphasizing specific subject 

products - 



6 . 



learnings prior to fragmenting into specincs ior 

the student will become familiar with the background of the 12th and 1^-h 
century European interest in colonies and trade, prior to studying the 
specific explorations. The extreme of the above approach would be a 
S sSS sten^y S-p sequencing of the material on Europe in the 12th and 
mh century in which concepts on European interest in trade and colonies 
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were fed to the student on a programmed basis, eventually leading through 
the various explorations. There objectives are based on different 
approaches to learning. 

7. *Objectives based on demands and needs of child - objectives using this 

emphasis usually have as their focus some developmental sequence (physical, 
emotional, or social) as their central organizer. Example: the student 

will express affection as well as receive affection. The behavior of 
expressing affection is developmentally more advanced than simply 
receiving affection. Example: the student will cooperate with another 

student on taking turns in using a game. If this objective is to be 
taught, it is usually sequenced with other objectives according to the way 
most children develop . 

8. *Task analysis — the materials have been developed into specific tasks 
for the learner which have behavioral requirements that suggest a 
sequence for presentation and which allow an observer to determine if the 
learner accomplishes the task. 

9. *Errorless discrimination - the tasks are sequenced in such a manner that 
the student should move from step to step without making errors. This 
technique is used in some types of programmed instruction. 

10. * Figure— ground — the organization of materials, frequently perceptual in 
nature, in a field so that one stands out in a distinct way (figure) and 
the rest remains in the background (ground) . Figure— ground organization 
can be used with other characteristics such as sounds, where one sound 
is heard over and above a background of others. 

11. *7o an affective response system - where recognition is given to different 
levels of attitudes, from the simplest of merely attending to an object, 
to the building up of complex attitudes which predispose one’s behavior 
toward a wide range of stimuli; e.g. enjoying a variety of forms of music. 

12. interrelationships of a subject — where the subject matter contains a 

logical relationship of concepts and processes. Example: adding must be 

mastered prior to multiplying. The local community is studied prior to 
more distant entities of state or federal government. 

13. *Positive reinforcement and programmed sequence — where the material has 
been developed into small steps that lead the learner toward a larger 
concept through a sequence that permits the learner to receive frequent 
reinforcement through knowledge of right answers . 

14. *0pen ended development of generalization — the instructional sequence is 
purposely quite open; e.g. letting the learner try out many possibilities 
and alternatives before arriving at a generalization. 

15. *Advanced organizers (cognitive) — a framework of key concepts, crucial 

to understanding and relating concepts of the larger body of material, are 
strategically placed in the sequence, forming an ideational ladder to 
which other material can readily be related. In some materials a short 
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summary preceding the main body of instructional material delineates the 
key concepts or stresses their relationship to other concepts known by 
tha learner, thus serving as advance organizers through the ideational 
anchors it gives to the learner for organizing, relating and remembering 
the new materials . 

16. *Modes of transaction - a transaction is the interaction of a learner and 
stimuli in this context consisting of instructional materials. A mode 
is the channel that is used. Is the student asked to passively view, 
manipulate, verbally organize? Is the teacher an important part of the 
mode through exercising control over the learner ’s channels of transaction? 
Is the student free to seek out channels of transaction or are they 
chosen for him? These are questions which must be answered when setting 
up modes of transaction (methodological) to be used with instructional 
materials . 

17. *Teacher-centric method - the teacher is largely responsible for choosing 
and directing the mode of transaction for the learner. Teacher-centric 
modes of transaction usually prescribe that the “teacher will . • •" and 
are predicated on obtaining specific learner responses. 

18. *Pupil-centric method - the learner is responsible for choosing the modes 
of transaction with the instructional material and is frequently lert to 
evaluate and revise his behavior toward materials without teacher 
supervision . 

19 . *Psy chomo tor skills — muscular or motor skills which require manipulation 
of mater ial or objects. The ability to stack blocks is a psychocsotor 
skill . 

20. *Af fee tive response — responses which emphasize feelings, emotion, or 
degree of acceptance or rejection stemming from internal attitudznal 
sets. Such responses may be labelled attitudes, biases, interests, etc. 

21. *Norm referent evaluation - judging a learner’s performance by what other 
known groups of learners do on the same tasks . Achievement test scores, 
aptitude tests and mental test scores report their results in norm 
referent ter ms - The statement "This particular learner scored at 4th 
grade level" is using a norm referent evaluation of the learner’s 
performance . 

22. ^Criterion referent evaluation — the learner is judged on his ability to 
do a specified task or demonstrate the behavior appropriate to the task. 

The learner is judged on whether he can or ca nno t demonstrate the 
appropriate behavior that signifies task accomplishment and is not judged 
by comparison of his performance with another group of learners. 
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